DALL·E Mini
Contents
DALL·E Mini#
This is an simple way of creating DALL·E Mini artworks for generative artists.
Note
Install ekorpkit package first.
Set logging level to Warning, if you don’t want to see verbose logging.
If you run this notebook in Colab, set Hardware accelerator to GPU.
Check your jaxlib version and install the appropriate version. for example,
pip install “jax[cuda11_cudnn82]” -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
!pip install ekorpkit[art] exit()
Prepare Environment#
%config InlineBackend.figure_format='retina'
%load_ext autotime
%load_ext autoreload
%autoreload 2 # reload modules before executing user code, set to 0 to disable
from ekorpkit import eKonf
eKonf.setLogger("WARNING")
eKonf.set_cuda(device="4,5")
print("version:", eKonf.__version__)
is_notebook = eKonf.is_notebook()
is_colab = eKonf.is_colab()
print("is notebook?", is_notebook)
print("is colab?", is_colab)
if is_colab:
eKonf.mount_google_drive()
eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")
INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:4)', 'A100-SXM4-40GB (id:5)']
INFO:ekorpkit.base:Setting EKORPKIT_WORKSPACE_ROOT to /content/drive/MyDrive/workspace/
INFO:ekorpkit.base:Setting EKORPKIT_PROJECT to ekorpkit-book
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
version: 0.1.40.post0.dev3
is notebook? True
is colab? False
('/content/drive/MyDrive/workspace', 'ekorpkit-book')
time: 2.25 s (started: 2022-11-01 06:49:16 +00:00)
Create a dalle-mini instance#
cfg = eKonf.compose("model/dalle_mini")
dalle = eKonf.instantiate(cfg)
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
INFO:ekorpkit.base:setting environment variable CACHED_PATH_CACHE_ROOT to /content/drive/MyDrive/workspace/.cache/cached_path
INFO:ekorpkit.base:setting environment variable KMP_DUPLICATE_LIB_OK to TRUE
wandb: Downloading large artifact mega-1-fp16:latest, 4938.53MB. 7 files... Done. 0:0:12.3
Some of the weights of DalleBart were initialized in float16 precision from the model checkpoint at /tmp/tmp833ehtl0:
[('lm_head', 'kernel'), ('model', 'decoder', 'embed_positions', 'embedding'), ('model', 'decoder', 'embed_tokens', 'embedding'), ('model', 'decoder', 'final_ln', 'bias'), ('model', 'decoder', 'layernorm_embedding', 'bias'), ('model', 'decoder', 'layernorm_embedding', 'scale'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_0', 'k_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_0', 'out_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_0', 'q_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_0', 'v_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_1', 'k_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_1', 'out_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_1', 'q_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'FlaxBartAttention_1', 'v_proj', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'GLU_0', 'Dense_0', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'GLU_0', 'Dense_1', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'GLU_0', 'Dense_2', 'kernel'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'GLU_0', 'LayerNorm_0', 'bias'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'GLU_0', 'LayerNorm_1', 'bias'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'LayerNorm_0', 'bias'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'LayerNorm_1', 'bias'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'LayerNorm_1', 'scale'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'LayerNorm_2', 'bias'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'LayerNorm_3', 'bias'), ('model', 'decoder', 'layers', 'FlaxBartDecoderLayers', 'LayerNorm_3', 'scale'), ('model', 'encoder', 'embed_positions', 'embedding'), ('model', 'encoder', 'embed_tokens', 'embedding'), ('model', 'encoder', 'final_ln', 'bias'), ('model', 'encoder', 'layernorm_embedding', 'bias'), ('model', 'encoder', 'layernorm_embedding', 'scale'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'FlaxBartAttention_0', 'k_proj', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'FlaxBartAttention_0', 'out_proj', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'FlaxBartAttention_0', 'q_proj', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'FlaxBartAttention_0', 'v_proj', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'GLU_0', 'Dense_0', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'GLU_0', 'Dense_1', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'GLU_0', 'Dense_2', 'kernel'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'GLU_0', 'LayerNorm_0', 'bias'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'GLU_0', 'LayerNorm_1', 'bias'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'LayerNorm_0', 'bias'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'LayerNorm_1', 'bias'), ('model', 'encoder', 'layers', 'FlaxBartEncoderLayers', 'LayerNorm_1', 'scale')]
You should probably UPCAST the model weights to float32 if this was not intended. See [`~FlaxPreTrainedModel.to_fp32`] for further information on how to do this.
wandb: Downloading large artifact mega-1-fp16:latest, 4938.53MB. 7 files... Done. 0:0:13.6
time: 50.2 s (started: 2022-11-01 06:49:18 +00:00)
Generate an image#
text_prompts = "a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape"
batch_name = "cave-jeju"
time: 835 µs (started: 2022-11-01 06:50:09 +00:00)
results = dalle.imagine(
text_prompts,
batch_name=batch_name,
num_samples=8,
show_collage=False,
)
8 samples generated to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju
text prompts: ['a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape']
time: 1min 15s (started: 2022-11-01 06:50:09 +00:00)
results
{'image_filepaths': ['/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0000.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0001.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0002.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0003.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0004.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0005.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0006.png',
'/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/dalle-mini/cave-jeju/cave-jeju(0)_0007.png'],
'config_file': 'cave-jeju(0)_config.yaml',
'config': {'batch_name': 'cave-jeju',
'text_prompts': ['a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape'],
'batch_num': 0,
'num_samples': 8,
'gen_top_k': None,
'gen_top_p': None,
'temperature': None,
'cond_scale': 10.0,
'set_seed': 'random_seed',
'seed': 2475039512,
'resume_run': False,
'run_to_resume': 'latest',
'show_collage': False}}
time: 2.22 ms (started: 2022-11-01 06:51:24 +00:00)
Collage generated sample images#
dalle.collage(
batch_name=batch_name,
batch_num=4,
ncols=4,
num_images=8,
show_filename=True,
fontcolor="white",
)
time: 2.08 s (started: 2022-11-01 06:51:44 +00:00)
Show configuration#
dalle.show_config(batch_name=batch_name, batch_num=4)
{'batch_name': 'cave-jeju',
'batch_num': 1,
'cond_scale': 10.0,
'gen_top_k': None,
'gen_top_p': None,
'num_samples': 8,
'resume_run': False,
'run_to_resume': 'latest',
'seed': 1011150451,
'set_seed': 'random_seed',
'show_collage': False,
'temperature': None,
'text_prompts': ['a living tiger inside a gorgeous cave in a futuristic Jeju '
'island, relaxing night, watercolour on the ceiling, glowing '
'on glass, light art, dark, dark, gorgeous, dark, bright in '
'an abandoned village in a futuristic alien nightscape']}
time: 5.91 ms (started: 2022-11-01 06:51:50 +00:00)